Add multi-GPU vLLM CAA steering hook#694
Conversation
|
Hello, thank you for your contribution. To maintain consistency across our repositories, we have reimplemented this in PR #707. |
Thanks for reviewing #694 and adapting the design into #707. I reviewed the merged implementation, and I understand the direction: #707 generalized the worker-side activation-add hook into EasyEdit2’s apply/inference stage for multiple activation-addition methods, while keeping vector generation in-process. I would be happy to contribute follow-up improvements on top of #707, especially:
I also want to raise one attribution point. Since #694 was the original prototype/design proposal for the worker-side CAA/vLLM hook, and the maintainers said its logic was manually adapted into #707, I think this contribution should be properly recognized if it materially contributed to the core merged method. I am not asking to rewrite history, but I would appreciate a short acknowledgement in #707, #694, or the relevant vLLM documentation. |
|
Thank you for raising this issue and helping us identify the problem. We appreciate your contribution and will acknowledge your support in the contributor section. You’re also welcome to submit additional code in the future. For code contributions, our main requirements are that they align with the existing framework, maintain simplicity, and avoid introducing overly AI-specific code. |
Summary
This PR adds a lightweight CAA activation-add hook for vLLM inference, including tensor-parallel worker support.
The hook lets EasyEdit apply already-computed CAA vectors to vLLM-loaded models by installing activation additions on selected decoder layers inside each worker process.
What Changed
Added
steer/vllm_caa_hooks.pyAdded
tests/test_vllm_caa_hooks.pyAdded
examples/vllm_caa_gpu_e2e.pyAdded
docs/vllm_caa_multigpu_hook.mdValidation
Result:
Result: passed.
Result: passed.
Optional GPU smoke command:
Notes
The larger steering-effect consistency experiments used during development are not included in this PR. This keeps the runtime contribution focused on the multi-GPU vLLM CAA hook and lightweight validation.
Related Issue
Closes #695